In a video coding system, the video data are subject to various processing such as prediction, transform, quantization, deblocking, and adaptive loop filtering. Along the processing path in the video coding system, certain characteristics of the processed video data may be altered from the original video data due to the operations applied to video data. For example, the mean value of the processed video may be shifted. Intensity shift may cause visual impairment or artifacts, which is especially more noticeable when the intensity shift varies from frame to frame. Therefore, the pixel intensity shift has to be carefully compensated or restored to reduce the artifacts. Some intensity offset schemes have been used in the field. For example, an intensity offset scheme, termed as sample adaptive offset (SAO), classifies each pixel in the processed video data into one of multiple categories according to a context selected. The conventional SAO scheme is only applied to the luma component. It is desirable to extend SAO processing to the chroma components as well. The SAO scheme usually requires incorporating SAO information in the video bitstream, such as partition information to divide a picture or slice into blocks and the SAO offset values for each block so that a decoder can operate properly. The SAO information may take up a noticeable portion of the bitrate of compressed video and it is desirable to develop efficient coding to incorporate the SAO information. Besides SAO, adaptive loop filter (ALF) is another type of in-loop filter often applied to the reconstructed video to improve video quality. Similarly, it is desirable to apply ALF to the chroma component as well to improve video quality. Again, ALF information such as partition information and filter parameters has to be incorporated in the video bitstream so that a decoder can operate properly. Therefore, it is also desirable to develop efficient coding to incorporate the ALF information in the video bitstream.